The IRMA Project: A State of the Art Report on Content-Based Image Retrieval in Medical Applications

The objective of this work is to develop a general structure for semantic image analysis that is suitable for content-based image retrieval in medical applications and an architecture for its efficient implementation. Stepwise content analysis of medical images results in six layers of information modeling (raw data layer, registered data layer, feature layer, scheme layer, object layer, knowledge layer). Medical expert knowledge is incorporated into several layers. In the registered data layer, a reference database with 10,000 images categorized according to the image modality, orientation, body region examined, and biological system imaged is used. By means of prototypes in each category, identification of objects and their geometrical or temporal relationships are handled in the object and the knowledge layer, respectively. Depending on the complexity of the query, it is processed on the higher layers starting with the scheme layer, where a hierarchical blob representation of image content is provided. Here, local image similarity is assessed by graph matching. The multilayer processing is implemented using a distributed system designed with only three core elements: (i) the central database holds program sources, processing scheme descriptions, images, features, blob trees, and administrative information about the workstation cluster; (ii) the scheduler balances distributed computing by addressing daemons running on all connected workstations; and (iii) the web server provides graphical user interfaces for data entry and retrieval, which can be easily adapted to a variety of applications for content-based image retrieval in medicine. Since manual labeling of reference data is still in progress, the system was used so far for processing primitive queries, i.e. queries regarding the category. However, since all feature transformations in all semantic layers are based the same implemented mechanism, this is sufficient to validate the overall system concept. The leaving-oneout experiments were distributed by the scheduler and controlled via corresponding job lists. The experiments have shown that the IRMA framework offers transparency regarding the viewpoint of a distributed system and the user, such as (i) location and access transparency for data and program sources; (ii) replication transparency for programs in development; (iii) concurrency transparency for job processing and feature extraction; (iv) system transparency at method implementation time; and (v) job distribution transparency when issuing a query. The proposed architecture is suitable for content-based image retrieval in medical applications. It improves current picture archiving and communication systems that still rely on alphanumerical descriptions, which are insufficient for image retrieval of high recall and precision.